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ABSTRACT: Let G be a complex simply-connected semisimple Lie group and 
let g = LieG. Let fj = n_ + f) + nbea triangular decomposition of g. One readily 
has that Cent£/(n) is isomorphic to the ring S^n)" of symmetric invariants. Using 
the cascade B of strongly orthogonal roots, some time ago we proved that <S , (n) n is a 
polynomial ring C[£i, . . . , £ m ] where m is the cardinality of B. Using this result we 
establish that the maximal coadjoint of N = exp n has codimension m. 

Let b = t) + n so that the corresponding subgroup B is a Borel subgroup of G. 
Let I = rankg. Then in this paper we prove 

Theorem. The maximal coadjoint orbit of B has codimension £ — m so that the 
following statements are equivalent: 

(1) — 1 is in the Weyl group of G ( i.e., I = m) 

(2) B has a nonempty open coadjoint orbit. 

Remark. A nilpotent or a semisimple group cannot have a nonempty open 
coadjoint orbit. Celebrated examples where a solvable Lie group has a nonempty 
open coadjoint orbit are due to Piatetski-Shapiro in his counterexample construction 
of a bounded complex homogeneous domain which is not of Cartan type. 

In a subsequent paper we will apply the results above to a split real form Gr. If 
I = 1 this gives rise (E. Stein) to the Hilbert transform. 
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1. Introduction 
Part A: Recollection of some of the relevant results of [K] 

1.1. Let G be a complex semisimple Lie group and let g = LieG. Let g = n_ + F) + n 
be a triangular decomposition of g and let I be the rank of g. Let b = f) + n and 
b_ = n_ + f). Let B,H,N be the subgroups of G corresponding, respectively, to 
b, f) n, so that B is a Borel subgroup of G and one has the familiar semidirect product 
B = H k N. 
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Let /C be the Killing form (x, y) on g. We may identify the dual b* of b with 
b_, where v E b* and z E b; then (u, z) = (v, z). With the obvious similar definitions 
the dual of n* is identified with n_. Let $(, : g — > b_ be the projection defined by 
the decomposition g = b_ © n, and let $ n : g — > n_ be the projection defined by the 
decomposition g = n_ © b. The coadjoint representation Coadf, of B may be given by 

CoaA b {g)v = $ b Adg v, 

where g E B and v E b_. For simplicity we will write g»v for Coadf, (g)v. The 
coadjoint representation Coad n of N is given similarly where n replaces b, n_ replaces 
b_, N replaces B and • by • . 

In order to prove the open B coadjoint theorem mentioned in the abstract, we will 
need the results of [K] . The results of [K] depend upon definitions and properties of the 
cascade of orthogonal roots. These will be used freely now, but for the convenience of 
the reader some of the definitions will be here recalled in Part A of the Introduction. 

The Killing form /C induces a nonsingular bilinear form [ji, v) on t)*. Let Act)* 
be the set of roots corresponding to (f), g). For each ip E A let E g be a correspond- 
ing root vector. The root vectors can and will be chosen so that (e^, e- v ) = 1 for all 
roots (p. If s C g is any subspace stable under ad t) let 

A(s) = {v? E A | e v E s}. 

The set A + of positive roots is then chosen so that A + = A(n), and one puts A_ = 
-A + . 

Let B C A + be the cascade of orthogonal roots; See §1 in [K]. Then card£> = m 
where m is the maximal number of strongly orthogonal roots. See Theorem 1.8 in [K]. 
As in §2.1 in [K], let r = X^eB ^ e P so that r is an m-dimensional commutative Lie 
subalgebra of n. Let R be the m-dimensional commutative unipotent subgroup of A^ 
corresponding to r. As in §2.1 of [K], let r_ C n_ be the span of e-p for f3 E B. For 
any z E r_, (3 E B, let ap(z) E C be defined so that 

z=^2ap(z)e-p (1.1) 

and let 

r! = {ret_ ap(T)^0, V/3 E B). 
As an algebraic subvariety of n_, clearly 

X r*j /(pX\m 
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For any v G n_ let O v be the iV-coadjoint orbit containing v. Let N v d N he 
the coadjoint isotropy subgroup at v and let n„ = Lie N z . Since the action is algebraic, 
N v is connected and as iV-spaces 

O v N/N v . 

The following result appears as Theorems 2.3 and 2.5 in [K]. 

Theorem 1.1. Let r G t*. Then (independent of r) N T = R so that (1.1) 
becomes 

O t = N/R. (1.2) 

Furthermore if r, t' G t* are distinct, then O t fl O r ' = so t/iat one /ios a disjoint 
union 

N-t*=U TEx xO T . (1.3) 

Since S normalizes iV there is a natural action on B on n_. We refer to this as 
the n_ action of B. Explicitly since the latter extends Coad n we denote the n_ action 
of B of b G B on v G n_ by b ■ v, and note that 

b-v = $ n oAdbv. (1.4) 

The Zariski open subvariety C r_ is stable under the n_ action of H C B 
and in fact H operates transitively on so that is isomorphic to a homogeneous 
space for H. 

The action of if on extends to an action of H on the corresponding set {O r , r G 
r^} of iV-coadjoint orbits. Since H normalizes N the following statement is obvious. 
See §2.2 in [K]. 

Proposition 1.2. For any r G and a G H one has 

OAda(r) = a ■ T . 

Let 

X = U T€r x O t (1.5) 

so that, by (1.3), the union (1.5) is disjoint. Furthermore since B = NH we note that 
X is an orbit of n_ action of B and hence is Zariski open and dense in its closure. 
The following is one of the main theorems in [K]. See Theorem 2.8. in [K]. 
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Theorem 1.3. X is Zariski dense in n_ so that 



X = n_. 



(1.6) 



Let W be the Weyl group of g operating in \) and h*. Reluctantly submitting to 
common usage, let w a be the long element of W. For any (3 G B let sp G W be the 
reflection defined by /3. Since the elements in B are orthogonal to one another, the 
reflections sp evidently commute with one another. The long element w a of the Weyl 
group W is given in terms of the product of these commuting reflections. In fact one 
has (see Proposition 1.10 in [K]). 

Proposition 1.4. One has 



noting that the order of the product is immaterial because of commutativity. 

Remark. Note that it is immediate from Proposition 1.4 that — 1 G IF if and 
only if m = £. 



1.2 With respect to the coadjoint structure of N one has 

Theorem 1.5. A maximal coadjoint orbit of N has codimension m in n* where 
m is the maximal number of strongly orthogonal roots. In particular it has codimension 
£ (the rank of g) in n* if and only if —1 G W. 

We now deal with the coadjoint representation Coadt, of the Borel subgroup B. 
As noted in §1.1, we have identified b* with b_. For any w G b_ let B w be the 
B-coadjoint isotropy group of B at w and let b w = LieS^. We refer to b w as the 
B-coadjoint isotropy algebra at w. Let O w C b_ be the S-coadjoint orbit of w so 
that 



Ow — B/B w . 

Of course h C b. Let f)° C h be the orthogonal subspace to CB C h* in f). 
Then 



n ^ 



(1.7) 



pes 



Part B. Statement of new results to be proved in $2. 



dimh° = £-m. 



(1.8) 
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Now let <£>[, : g — > t) be the projection defined by the triangular decomposition 
= n_ + h + n. Obviously any w G b_ can be uniquely written as 



w = v + x, 



(1.9) 



where v = & n w and x = Q^w. It is immediate from Theorem 1.3 that b'_ is open in 
b_ where b'_ is defined by 



We will prove 

Theorem 1.6. Let w G r\ Then the B-coadjoint isotropy algebra of b at w is 
t)° . Furthermore O w is a maximal B-coadjoint orbit so that i — m is the codimension 
of any maximal B-coadjoint orbit. Furthermore O y is a maximal B-coadjoint orbit of 
B for any y G b'_. 

As a consequence we now have 

Theorem 1.7. A Borel subgroup of a complex semisimple Lie group has a 
nonempty (and hence unique) open coadjoint orbit if and only if m = I, that is if 
and only if —I is in the Weyl group W . Furthermore, explicitly, the unique nonempty 
open coadjoint orbit of B is b'_ . 



2.1. The proof of Theorem 1.5 depends on the density of X and standard argu- 
ments about dimensions of orbits. 

Proof of Theorem 1.5. Let v G X. Since dimR = m it follows from (1.2) 
and (1.5) that the codimension of O v in n_ is m. But if w G n_, then by (1.6) there 
exists a sequence Vk G X which converges to w. But then by the compactness of 
subspaces of dimension m there exists a subsequence Vk> such that n v , converges to 
an m-dimensional subspace of n w . Hence the codimension of O w in n_ is greater than 
or equal to m. QED 

Recall (1.4). 

Theorem 2.1. Let w£t!. Then the isotropy subalgebra of b at w with respect 
to the n_ action of B is h° + r. 



b'_ = 



{w G b- | $ n w G X}. 



(1.10) 



2. Proofs of stated results and additional results 
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Proof. Let c w be the isotropy subalgebra of b at w with respect to the tr- 
action of B. Since B = NH and since the n_ action of B extends Coad n clearly 
(f)° + 1) C c w . But 

B-w = X (2.1) 
by Proposition 1.2 and Theorem 1.1. But dim (t)° + r) = £ since clearly 

dimf)° = £-m (2.2) 

and dimr = m. But dimX = dimn by (1.6). Thus dimc TO = I. This implies 
f)° + r=c w . QED 

Comparing Coadf, with the n_ action of £?, we note that if w G n_ and b G B 
(recall the notation of §1.1.), one has 

b»w = b ■ w + b»w (2-3) 

This is immediate since = $ n + <frf,. This latter sum also yields the following 
infinitesimal analogue of (2.3). Let x G b and to G n_; then 

x = x ■ w + »w (2-4) 

where = $b and x -w = 3> n [x,w]. But now (2.1) and (2.3) imply 

Proposition 2.2. Let w G t*. TTien 

$ n S.«; = X (2.5) 

We also note that the fixed point set b^ of the coadjoint action of B is immediately 
given by 

b? = f). (2.6) 

One now has 

Theorem 2.3. Letw G t*. TTien £/ie isotropy algebra, b w , ofb atw, with respect 
to the coadjoint action of B is f)° so that, by (2.2), t — m is the codimension of the 
coadjoint orbit B»w. 

Proof. Let x G b_. Then, by Theorem 2.1 and (2.4) one has x G b w if and only 
if (1) x G f)° + r and (2) $ij[:r,w] = 0. But clearly h,° C b w . Thus to prove b w = f)° 
it suffices to show that if 



7^ x G r then [x, tu] 7^ 0. 
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(2.7) 



But since the all the coefficients ap(w) in (1.1) are not zero and the set {/3 y G I) | (3 G 
B} are linearly independent, one has (2.7). QED 



2.2. As an immediate consequence of (2.6) one has 

Proposition 2.4. Let w G b_ and z G f); then for any v G O w one has v + z G 

O w + z and the map 

O w ->■ u i-> v + 2 (2.8) 

zs a B -isomorphism of B-coadjoint orbits. 

As an immediate consequence of Proposition 2.4, one has 

Proposition 2.5. Let w G b_ and z G f). T7ien 

b w = b w+z (2.10) 

so t/iat obviously in particular 

dimb TO = dimb w+z . (2.10) 

From the open density of X in n_, it is obvious, recalling (1.10), that b'_ is 
Zariski open and dense in b_. 

Theorem 2.6 For any y G b'_ the dimension of b y in b is t — m where we recall 
that m is the cardinality of a maximal set of strongly orthogonal roots. Furthermore 
O y is a maximal B-coadjoint orbit and has codimension I — m in b_. That is, the 
codimension of any B-coadjoint orbit O is greater than or equal to £ — m. 

Proof. Let v = so that v G X. But then by (2.1) and (2.3) there exists 
wet! and b G B such that <& n b*w = v. But by conjugacy and Theorem 2.3, 
dimbfw = £ — m. But if z = y — b»w, then z G f). But then dim b y = t — m by 
Proposition 2.5. But then using the limit argument in the proof of Theorem 1.5, one 
has dim b x > £ — m for any x G b_. QED 

Applying Theorem 2.6 to the case where £ = m we come to the statement and 
proof of our main theorem. 

Theorem 2.7. Let G be a complex semisimple Lie group and let B be a Borel 
subgroup of G. Then B has a (necessarily unique) nonzero open coadjoint orbit, O, 
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if and only if — 1 is in the Weyl group of G, or equivalently, there exists I orthogonal 
roots where t is the rank of G. Furthermore in the notation above 

O = b'_ (2.11) 

where b'_ is given by (1.10). In particular t* C O. (See §1.1.) 

Proof. By Theorem 2.6 B has an open coadjoint orbit O in b_ if and only if 
m = I. The orbit is necessarily unique since the action of B is linear algebraic and b_ 
is affine irreducible. But now b'_ C O by Theorem 2.6. On the other hand, O C b'_ 
by Proposition 2.2. This proves (2.11). QED 

Remark 2.8. If G is simple, then B has a nonempty open coadjoint orbit 
if and only if q is of type Bg, Cg, Dg with £-even, G2, -£4, £7, Eg. Except for the 
case Dg, with £-even, the other cases follow from the fact that there exists no outer 
automorphism. This fact readily implies —1 is in the Weyl group. 
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